System and method for processing network packet flows

ABSTRACT

A network node includes a classify engine interfaced with the Internet. The classify engine accepts packets from the Internet and determines classification information for each packet. A process engine is interfaced with the classify engine, and has ports, each port being associated with a function. A controller is interfaced with the classify engine and the process engine. The controller programs the classify engine with a dataflow program to route each packet to a predetermined port of the process engine based on the classification information of the packet.

The present application is a Divisional Application of commonly assignedand U.S. patent application Ser. No. 12/553,664, filed on Sep. 3, 2009,now U.S. Pat. No. 8,125,905 which claims priority from and is acontinuation of U.S. application Ser. No. 11/473,885, now U.S. Pat. No.7,633,868, filed on Jun. 23, 2006, which is a Divisional of U.S. Pat.No. 7,095,715, U.S. application Ser. No. 09/897,189 filed on Jul. 2,2001, which are incorporated by reference in their entireties.

TECHNICAL FIELD OF THE INVENTION

This invention relates generally to computer networks, and morespecifically relates to a system and method for processing packet flowsthrough a network.

BACKGROUND OF THE INVENTION

The Internet has grown in popularity largely because it provides asimple and uniform underlying packet protocol for exchanging data thatin turn enables more complex applications to occur. The relativesimplicity of the Internet has lead to an explosion of growth in datatraffic for business and personal usage. An industry of Internet serviceproviders (ISP) has developed to provide access to the Internet forbusinesses and homes. These ISPs have invested heavily in infrastructurewith the general goal of improving data transfer rates for customer endpoints. Thus, ISPs have evolved from low speed analog modem Internetconnections provided through dial-up service across the plain oldtelephone system (POTS) into broadband access provided by cable modems,DSL, and ADSL that are capable of transferring data at substantiallyhigher rates.

Internet access offered by ISPs has become a commodity service withdifferent ISPs typically offering similar access speeds and competing onprice points. However, as ISPs gain subscribers to their high-speedbroadband access services, the reliability of ISP service, i.e. theability to transfer data at predictable rates, available at end pointshas suffered and indeed is often less than the capacity of broadbandcustomer modems. For instance, surges in subscriber use tends to createbottlenecks that slow data transfer rates and use ISP infrastructureinefficiently. This difficulty relates generally to the Internet'sarchitecture which transfers data on a “best effort” basis in whichTCP\IP packets are generally transferred between routing points withoutprioritization. This “best effort” architecture is attractive for itssimplicity, but creates difficulties for ISPs who desire to distinguishthemselves based on the services offered compared to other ISPs.

One manner in which ISPs are working to distinguish themselves is tooffer subscribers different types of services. In general terms, aservice is the processing of certain data on the network in apredetermined manner with associated billing. For instance, one servicetypically offered by telephone network providers is call waiting, inwhich customers receive notice of an incoming telephone call when thecustomer is on the phone for a monthly fee. A typical service offered byISPs is e-mail service, although e-mail service is not typically billed.Typically e-mail service is provided by routing e-mails to an ISP serverthat stores the e-mails until the subscriber retrieves them.

More complex services are generally difficult to implement on a besteffort network architecture, such as the Internet, since best effortnetworks are generally designed to route packets to destinations on afirst come first served basis. An ISP that seeks to provide a newservice to its subscribers has to design and install hardware andsoftware that generally require a considerable capital investment andtime to develop. Even after investing considerable time and capital inthe development of a new service, subscriber needs and technology oftenadvance more rapidly than a service deployment cycle, frequently leavingnewly deployed services out of date before the services become a revenuesource or market differentiator for the ISP. Thus, ISPs who attempt tolead the market by developing and offering new services faceconsiderable risk if a large investment in a new service fails to resultin greater revenue or customer satisfaction.

SUMMARY OF THE INVENTION

Therefore a need has arisen for a system and method which processesnetwork packet flows in a best effort network with scalable and flexiblehardware and software architecture.

A further need has arisen for a system and method which supports rapiddeployment of services in a best effort network with reduced expense andcomplexity.

In accordance with the present invention, a system and method isprovided that substantially eliminates or reduces disadvantages andproblems associated with previously developed systems and methods forprocessing network packet flows in a best effort network. A network nodeprovides a data path that performs repetitive operations at high speedsto classify, process and shape packets according to a slower speedcontrol path that performs complex logic operations associated withpredetermined services. Processing blades interfaced with the data flowreceives predetermined packets based on classification and modifies thepackets for transmission across the network.

More specifically, packets received by the network node are classifiedaccording to a set of classification rules. The classification rulessort the packets into packet groups and each packet group is sent to apredetermined input port of a modify\process engine. The modify\processengine performs multithreaded processing with a function associated witheach input port. This highly parallel implementation in which internalstate is not shared among functions allows the network node switch toperform repetitive operations at high speeds for processing packetflows. The modify\process engine sends packets to a shaper engine whichcontrols the transmission of the packets with queuing disciplines fortransmission to the network. An accountant engine tracks statistics inthe classify, modify\process and shaper engines for control and billingpurposes. A controller programs the classify, modify\process, shaper andaccountant engines to achieve a desired packet prioritization and flow.For instance, the controller installs or modifies rules in the classifyengine, installs or modifies functions in the modify\process engine,establishes queues in the shaper engine and determines the variables ortracking in the accountant engine.

In one embodiment, packet flows are controlled by packet processingbehaviors that are specified with a dataflow program and reflectiveprogramming. The dataflow program specifies interconnected objects forthe data to flow through. For instance, one packet processing behaviorof a service in a dataflow representation creates a queue for a new TCPconnection. The dataflow program detects SYN packets of new TCPconnections and, when a new connection is detected, a new dataflowprogram is constructed using reflective programming that containselements to detect the host/port quadruple of the new flow. Thus,bandwidth may be dynamically assigned for new connections.

The modify\process engine includes an interface with a blade bus adaptedto accept processor blades programmed to modify packets. Each processorblade performs a function on packets received and then forwards themodified packet to the shaper engine for transmission on the network.Each processing blade is designed for a specific function to enablecomplex operations with minimal delay. The processing blade architectureprovides a scalable broadband network node that is easily adapted tosupport new services with reduced complexity and increased speed. Forinstance, a broadband network node programmed to classify packets forsupporting a voice over internet protocol (VOIP) is adapted to support aconference call service by interfacing a processing blade with theprocessor blade bus so that packets classified as conference call VOIPpackets are mixed as audio streams with the processing blade.

The present invention provides a number of important technicaladvantages. One important technical advantage is the inexpensive andrapid deployment of new and modified services in a best effort network.The classification, modification and shaping of packets is performed atline speed by software residing on function specific hardware.Flexibility is provided to add and modify services through softwareprogramming and the addition of appropriate hardware processing blades.Thus, the broadband network node provides a flexible and scalablesolution for routing packets in a best effort network according to thepacket's associated applications.

Another important technical advantage of the present invention is thatbandwidth is dynamically allocated for packet flows using reflectiveprogramming techniques in the context of dataflow programs. This allowsthe network to adapt as the type of data across the network changes overtime, resulting in improved efficiency and flexibility in bandwidthusage.

Another important technical advantage of the present invention is thatit provides separate mechanisms for a data path and a control path inrouting packets. The data path is highly parallel, processing eachpacket flow independently to perform repetitive operations at highspeeds for routing packets including complex modifications of thepackets. The control path looks across data flows to implement the datapath with the parallel processing so that complex routing operations areperformed in an efficient manner at line speeds.

BRIEF DESCRIPTION OF THE DRAWINGS

A more complete understanding of the present invention and advantagesthereof may be acquired by referring to the following description takenin conjunction with the accompanying drawings, in which like referencenumbers indicate like features, and wherein:

FIG. 1 depicts a block diagram of the logical architecture of abroadband network node; and

FIG. 2 depicts a block diagram of a hardware architecture for modifyingpackets with a processing blade architecture.

DETAILED DESCRIPTION OF THE INVENTION

Preferred embodiments of the present invention are illustrated in thefigures, like numerals being used to refer to like and correspondingparts of the various drawings.

Building a service for a network presents a substantial task which isoften time consuming and expensive. For instance, years often pass fromthe development and approval of a business case through the design andprovisioning of a service on a network. The conventional development andprovisioning of a service on a best effort network, such as the Internetor intranets that use TCP\IP, are difficult to design and deploy,typically requiring design from scratch and custom equipment. Even oncea service is deployed on a best effort network, modification of theservice over time presents a continuing problem.

One solution that eases the development and deployment of services on abest effort network is to deploy broadband network nodes on the edges ofthe network, such as the advanced traffic processor disclosed in U.S.patent application Ser. No. 09/875,639, entitled “A System and Methodfor Allocating Bandwidth Across a Network,” incorporated herein byreference. A broadband network node is a hardware device that processesnetwork packet flows at line speeds at the ingress and egress points ofan intranet. For instance, services are provisioned to the intranet byprogramming the broad band network nodes to classify, modify and shapepackets to accomplish the desired service.

Referring now to FIG. 1, a block diagram depicts the logicalarchitecture of a broadband network node 10 for provisioning andoperating a service on a best effort network. Generally, a service is apacket processing behavior or behaviors and associated billing ruleswhich provides value to subscribers to the service. Broadband networknode 10 provides packet processing behaviors through software thatclassifies, processes and shapes packet traffic flowing throughbroadband network node 10. Broadband network node 10 allows flexible andscalable deployment of services through software additions andmodifications and, in some cases, the addition of hardware modules, thatoperate the service with separate data path and control path mechanisms.A highly parallel data path processes packet flows independently toperform repetitive operations at high speeds which reduce the impact ofthe service on network data transfer rates, while the control pathperforms more complex logic at lower speeds to allow control of one ormore services by looking across data flows.

The data path mechanism is the flow of packets 12 through a classifyengine 14, a modify\process engine 16 and a shaper engine 18. The datapath mechanism uses a dataflow program which specifies a set ofinterconnected objects through which data flows and is processed. Anaccountant engine 20 tracks statistics in classify engine 14,modify\process engine 16 and shaper engine 18 to evaluate the data pathusage and for billing purposes. The control path mechanism isestablished by controller 22 which interfaces with classify engine 14,modify\process engine 16, shaper engine 18 and accountant engine 20.Controller 22 programs the dataflow programs of each of the engines. Forinstance, installing or modifying rules in classify engine 14,installing or modifying functions in modify\process engine 16,establishing queues in shaper engine 18 and determining the variables totrack in accountant engine 20. Controller 22 also controls the routingof the packets between the engines.

Classify engine 14 accepts inbound packets 12 and applies a set ofclassification rules in a well-defined order determined by controller 22so that the first matching rule defines the packet group. Classificationinto packet groups is based on any part of the packet, including thedatalink, network, and transport header fields as well as packet contentor payload. For example, classify engine 14 groups packets based onVCI\VPI\CLP (ATM fields), IP source or destination addresses, portnumbers, protocol type such as TCP or UDP, application protocol such asDNS, DHCP, HTTP, or the MIME content type such as text\html.

In one embodiment, classify engine 14 classifies groups of packets byflow statistics maintained by classify engine 14 or accountant engine20, such as flow lifetime, average bandwidth, and peak bandwidth. Forinstance, classify engine 14 finds the flow to which a packet belongs,updates the flow statistics and then uses the flow statistics toclassify the packet. In one application of this embodiment, a TCP flowgreater than a predetermined age, such as 30 seconds old, is sent to aspecific port.

In another embodiment, classify engine 14 creates a packet flow when apacket of a specific type of packet is detected, thus allowing flexibleallocation of bandwidth. The network processor runs a program, such as arules-based or a dataflow program, that is able to use reflectiveprogramming techniques to create a new reflective program able toclassify desired packets. For instance, a dataflow program classifiespackets according to the packet's control protocol, such as DHCP, DNS,FTP, SIP and other control protocols. The control protocol includes datathat provides a basis for defining a new packet processing behavior todefine a service, such port numbers, host names, user names, and IPaddresses. The dataflow program extracts relevant data from the controlprotocol and uses reflective programming to create a new dataflowprogram that contains those parameters to define a service thatclassifies packets with the newly created dataflow program running onthe network node.

One example of a service supported with reflective programming ofprograms includes a packet processing behavior of creating a new queuewhen a new TCP connection is established. Controller 22 programs adataflow program associated with classify engine 14 to watch for TCP SYNrequests associated with new TCP connections. When a SYN request isdetected, the request is passed on to the appropriate port ofmodify/process engine 16 for transmission and also used to construct anew dataflow program by applying reflective programming techniques.Parameters of the SYN packet, such as IP source and destination addressand port known as the host/port quadruple, are used to construct a newdataflow program. The new dataflow program creates a flow based on thehost and port quadruples of the new TCP connection. The new TCPconnection's dataflow program is added to classify engine 14 with anappropriate function added to modify/process engine 16. The reflectiveprogramming is accomplished by controller 22, which identifies a queuefor the new TCP connection.

Other examples of services provided by reflective programming includeservices for classifying, modifying and shaping packets with FTP, DHCPand DNS control protocol data. For instance, for services related to FTPdownloads, classify engine 14 monitors the FTP control channel until theparameters for the data connection, and then uses those parameters todynamically construct a new dataflow program that gives FTP connectionson the detected host port quadruple an appropriate class of service. Asanother example, the classify engine monitors DHCP requests andresponses to extract the MAC-IP mapping for constructing rules based onIP addresses. This allows more efficient services where DHCP is used todynamically assign IP addresses to computers of the network since 32 bitIP address are shorter than 48 bit MAC addresses. Similarly, theclassify engine may monitor DNS requests and responses to allow creationthrough reflective programming of a new dataflow program that processestraffic according to IP address information. This advantageously allowsdynamic creation of rules for a dataflow program based on DNS mappings,which is otherwise difficult to accomplish since the mapping of hostname to IP address varies across the Internet.

Further, the reflective programming technique is able to provideservices by creating dataflow programs with reflective programming thatare in turn able to create new dataflow programs through the applicationof additional reflective programming. One example of multi-layeredreflective programming is the monitoring of peer-to-peer downloads usinga directory lookup service. For instance, Napster allows users to lookupat a central directory the IP addresses and port numbers for computersstoring a specified song so that the user can connect to the computerand download the song. When a Napster request is detected, the Napsterconnection parameters are extracted and written into a first reflectiveprogram so that services, such as a blocking service, are provided bythe reflective program based on classification using the host/portinformation. This first reflective program can be created with anability to program a second reflective program. For instance, when thereflective program is created based on a Napster connection, the firstreflective program looks for directory look-ups of specific songs andcreates a second reflective program to block connection requests forcertain specific predetermined songs. Thus, the second reflectiveprogram controls specific traffic associated with the first reflectiveprogram to allow more precise control of network traffic, such asidentification of particular songs associated with a more generalNapster connection.

Once classify engine 14 sorts packets into packet groups, each packetgroups is sent to a specific input port of modify\process engine 16.Each input port of modify\process engine 16 has an associated function24 that operates on the packets received at the associated ports.Modify\process engine 16 is a highly scalable, multithreaded engine thatprocesses packets according to the set of functions associated with theinput ports so that internal state is not shared among functions, thusallowing a highly parallel implementation. Modify\process engine 16 isable to execute arbitrary programs on any packet to perform one of avariety of functions ranging from a non-operation in which the packet ispassed unchanged to encapsulation or encryption of the packets.

Modify\process engine 16 sends packets to shaper engine 18 whichcontrols the transmission of the packets to the network. Shaper engine18 uses a variety of queuing disciplines to route the processed packets.For instance, shaper engine 18 can discard packets that are associatedwith an unauthorized use, can perform weighted fair queuing (WFQ) andleaky bucket queuing. Queuing of classified and processed packets byshaper engine 18 allows a broadband network node 10 to control packetflows across the network to achieve desired services.

By programming controller 22 to process predetermined packets, broadbandnetwork node 10 provides a flexible way to accomplish complex controlwith software programming. Although conventional routers are able toperform routing operations based on header fields that are at predefinedoffsets and fixed size, such routing is generally unable to performcomplex classify and modify operations for enabling services. Forinstance, line speed operations on content of packets and high levelstate operations for multipacket flows or protocol interactions are notpractical with conventional routers. To achieve the line speed complexprocessing of packets outlined in the logical architecture depicted byFIG. 1, one embodiment of the present invention redirects packetsrequiring complex processing to function specific processing blades.

Referring now to FIG. 2, a block diagram depicts a hardware architecturefor a broadband network node 10 which is programmable to perform thelogical functions depicted by FIG. 1. Packets 12 flow to physicalinterface 26 and classify engine 14, which is a fast pattern processorfrom AGERE that is able to classify the packets. Once the packets areclassified, they pass to a modify\process engine 16 which performs theset of functions based on the classification of classify engine 14.Modify\process engine 16 is a hardware interface 28 that interfaces witha processor blade bus 30 and one or more processor blades 32. Processorblade interface 28 is an FPGA or ASIC that passes packets received fromclassify engine 14 to either processor blade bus 30 or shaper engine 18,which is a routing switch processor from AGERE. Processor bladeinterface 28 performs functions based on classification or forwardspackets to processor blades 32 based on classifications and thecomplexity of the function. For instance, to enable voice over internetprotocol, conference calls, a processing blade 32 with an audio mixeraccepts packets forwarded from processor blade interface 28 throughprocessor blade bus 30 and performs audio mixing functions for theconference call. The packets are then forwarded back to processor bladeinterface 28 and on to shaper engine 18 for transmission throughphysical interface 26 to the network.

One advantage of the processor blade architecture is that the broadbandnetwork node 10 is extremely scalable and able to add services byinserting a supporting hardware module. Complex operations on packetsassociated with processor blades do not slow the transfer rate of otherpackets since the processor blade architecture horizontally scales toone processing blade for each flow in the network. Some examples ofservices of varying degrees of complexity that may be performed withnetwork processors or with processor blades include:

Complexity (processing Operations power) Applications ClassifyModify/Process Account Low Routing MAC Addr Change MAC addr Number ofbytes Switching (scr/dest) Add/Modify MPLS transferred Firewall (basedon UO Addr field (layer 2.5) Number of bytes ports) (src/dest) Change IPaddress received VLAN mapping ATM VCI/VPI, CLP Change VCI/VPI Bytes persecond Tiered Services Type of Service Change Diffserv (average, peak,Content Broker (TOS) field priority field sustained) (based on IP addr)Transport (i.e., Change TOS field FAP UDP, TCP, RTP) Change Time to LoadBalancing Port number Live (TTL) field (scr/dest) Layer 2 ProtocolSwitching (i.e. ATM to Ethernet) Moderate Application to QoS Application(i.e., Add/Delete/Modify Number of sessions mapping HTTP, Napster, IP orprotocol Number of sessions Admission Control Realmedia) based Optionsfield by application (i.e., Content Broker on port number audio streams,VoIP) (based on URL) URL/Cookie Data Re-compute and Start/Stop times ofNetEx (bandwidth type (i.e. MIME apply padding sessions on demand)encoding) Re-compute and URL/Cookie Content checking apply Header and/ortracking (i.e., names in XML, Protocol Checksum email headers) ProtocolScraping (i.e., DHCP) High Content Broker CDN URL Encapsulation Contenttracking (w/redirection) redirection IPSec Encryption (i.e., VoD,Digital Rights Content Filtering Digital Rights Carnivore)ContentManagement Virus scanning Watermarking tracking (i.e., VoD, VirusDetection Digital Rights Carnivore) Carnivore Classification

By having these services selectively provided through softwareprogramming or a combination of programming and the insertion ofprocessing blades to achieve a desired performance level for a service.Processing performance scales linearly with the number of processingblades 32. Each processing blade is able to have either a generalpurpose CPU or specifically designed hardware for desired functions,such as encryption or audio mixing, so that new services are easilyadded with modular hardware. Some examples of general purpose orspecialized engines supported by processing blades 32 includeencryption, audio mixing, water marking, transcoding, filtering andsearching.

Although the present invention has been described in detail, it shouldbe understood that various changes, substitutions and alterations can bemade hereto without departing from the spirit and scope of the inventionas defined by the appending claims.

What is claimed is:
 1. A network node comprising: a classify engineinterfaced with the Internet, the classify engine to accept packets fromthe Internet and determine classification information for each packet; aprocess engine having a plurality of ports to receive the packets fromthe classify engine, each port having an associated function; and acontroller interfaced with the classify engine and the process engine,wherein the controller is to program the classify engine to route eachpacket to a predetermined port of the process engine based on theclassification information of the packet; wherein the controller is toprogram the classify engine with a dataflow program that determines theclassification information for the packets, and wherein the classifyengine is further to monitor DHCP requests and responses to extract MACand IP address mapping, and the controller is further to program theclassify engine with rules to control traffic with IP addressinformation.
 2. The node of claim 1, wherein the classify engine isfurther to detect packets associated with an FTP data stream and thecontroller is further to program the classify engine with a new dataflowprogram that classifies the FTP data stream.
 3. The node of claim 1,wherein the classify engine is further to monitor DNS requests andresponses to associate traffic with an IP address and the controller isfurther to program the classify engine with rules to control trafficwith IP address information.
 4. The node of claim 1, wherein thecontroller detects host and port quadruples of a new TCP connection andcreates a new dataflow program based on the host and port quadruples forthe new TCP connection.
 5. The node of claim 4, wherein the new dataflowprogram is provided to the classify engine, and the classify enginecreates a new flow based on the new dataflow program to classify packetsfor the new TCP connection.
 6. The node of claim 5, wherein a newfunction is added to the process engine for the new flow, and the newfunction is associated with a port of the process engine.
 7. The node ofclaim 1, wherein the node comprises a broadband network node processingpackets from the Internet.
 8. A method for routing Internet packets, themethod comprising: classifying, by a processor, the packets into one ormore packet flows according to classification rules; routing eachclassified packet flow to a predetermined input port of a process engineaccording to classification information of the packets, each input porthaving an associated function, so that the packets flow through theprocessing engine as a data path; programming the classification rulesand functions through a control path that looks across packet flows ofthe data path; detecting a new packet type; creating a dataflow programto classify the new packet type based on parameters of the new packettype; monitoring DNS requests and responses to associate traffic with anIP address; and programming a classify engine with rules to controltraffic with IP address information.
 9. The method of claim 8, whereinthe new packet type comprises a new TCP connection and creating adataflow diagram comprises creating the dataflow diagram based on hostand port quadruples for the new TCP connection.
 10. The method of claim9, comprising: creating a new flow based on the dataflow program toclassify packets for the new TCP connection.
 11. The method of claim 10,comprising: including a new function for processing packets for the newflow; and associating the new function with an input port of the processengine processing the packets for the new flow.
 12. The method of claim8, comprising: monitoring DHCP requests and responses to extract MAC andIP address mapping; and programming a classify engine with rules tocontrol traffic with IP address information.
 13. A system for processingpackets comprising: a classify engine executed by a processor andinterfaced with the Internet, the classify engine to accept packets fromthe Internet and determine classification information for each packet; aprocess engine having plural ports to receive the packets from theclassify engine, each port having an associated function; and acontroller interfaced with the classify engine and the process engine,wherein the controller is to program the classify engine to route eachpacket to a predetermined port of the process engine based on theclassification information of the packet; wherein the controller is toprogram the classify engine with a dataflow program that determines theclassification information for the packets, and wherein the controllerdetects host and port quadruples of a new TCP connection and creates anew dataflow program based on the host and port quadruples for the newTCP connection.
 14. The system of claim 13, wherein the new dataflowprogram is provided to the classify engine, and the classify enginecreates a new flow based on the new dataflow program to classify packetsfor the new TCP connection.
 15. The system of claim 13, wherein a newfunction is added to the process engine for the new flow, and the newfunction is associated with a port of the process engine.